Skip to content

Advanced configuration for Cadence Server #29

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 131 commits into
base: main
Choose a base branch
from

Conversation

CosminL-DEV
Copy link
Contributor

@CosminL-DEV CosminL-DEV commented Jun 6, 2025

Add comprehensive configuration support for Cadence server

Summary

This PR introduces a complete configuration system for the Cadence Helm chart, enabling users to customize all aspects of the Cadence server deployment through values.yaml.

Changes

  • Added comprehensive config section in values.yaml with support for:

    • Database configurations (MySQL, PostgreSQL, Cassandra) with TLS support
    • Elasticsearch integration for advanced visibility
    • Kafka integration for async workflows
    • Archival configuration (S3, GCS, Filestore)
    • Cluster and multi-cluster setup
    • Metrics and monitoring (Prometheus, StatsD)
    • Dynamic configuration management
  • Enhanced global configuration:

    • Added global.env and global.secretEnv examples for the use of S3 or GCP
    • Added global.tls for centralized TLS certificate management
  • ConfigMap integration: Configuration is automatically converted to YAML and mounted in server deployments

Benefits

  • Eliminates need for env variables for cadence configs
  • Provides type-safe configuration through Helm values
  • Supports all Cadence deployment scenarios (single cluster, multi-cluster, various backends)

Backward Compatibility

All changes are backward compatible. Existing deployments will continue to work with default configurations.


Test Checklist

  • Default Deployment - Basic deploy with defaults
    image
    image
    image

  • Secrets handled - Database Secrets handled correctly in configMap and application
    image
    image

  • Log Use Env - Check if the boolean allows to use different log level for each component.
    image
    image
    image
    image

Database Connectivity

  • MySQL basic connection - Basic connection with credentials
    Deployed Bitnami MySQL version 10.3.0 with appversion 8.0.37.
    image
    image
    image

  • MySQL with TLS - TLS encryption enabled
    Testing with latest version of mysql for auto-generation of tls, chart version: 13.0.2 and appversion 9.3.0
    Enabled tls generation in mysql helm chart
    image
    Using a debug pod that I will share in a comment I created the schema with TLS.
    I changed this cadence values for test it
    image
    image
    image
    Working fine:
    image
    image

  • PostgreSQL basic connection - Basic connection with credentials
    With schema job disabled, I started a debug pod that I will share in the PR for apply the schema manually. I just changed this values:
    First I deployed Bitnami Postgresql Helm chart 16.7.0 with app version: 17.4.0 and then this values:
    image
    image
    image

  • PostgreSQL with TLS - TLS encryption enabled
    Deployed postgresql with tls auto-generated
    image
    image
    Started schema with debug-pod that I will provide in a comment when it's finished all the test cases.
    Changed this values for testing:
    image
    image
    image
    And working correctly:
    image
    image
    IMPORTANT: I will let the mode example by default in TLS because PostgreSQL requires the permission to be restrictive.

  • Cassandra basic connection - Basic connection with credentials
    image
    With schema job disabled, I started a debug pod that I will share in the PR for apply the schema manually. I just changed this values:
    First I deployed Bitnami Cassandra Helm chart 10.4.0 with app version: 4.1.1 and then this values:
    image

  • Cassandra with TLS - TLS encryption enabled
    image
    Enable that in cassandra values for generate secrets with certs
    image
    Now the cadence values changed for SSL with Cassandra
    image
    image
    Setup job and cassandra included deployment disabled.
    Schema Setup was run manually without TLS, schema setup is for next PR so I wont focus on that right now.
    Working all correctly
    image
    image
    IMPORTANT: serverName value will be default to cassandra because if we let empty, we will have this error connection:
    2025/06/23 10:51:43 error: failed to connect to 10.100.185.52:9042 due to error: tls: failed to verify certificate: x509: cannot validate certificate for 10.100.185.52 because it doesn't contain any IP SANs

Advanced Features

  • Elasticsearch + Kafka integration - Advanced visibility with Elasticsearch
    Deployed Bitnami ElasticSearch helm chart 21.6.3 with appversion 8.18.0.
    Deployed strimzi kafka version 3.9.0
    Deployed Cassandra from previus tests.
    All runing correctly:
    image
    Values.yaml for testing cadence with elastic:
    image
    image
    Schema has been loaded manually, I will provide the commands in a commentary here but the next PR with schema jobs will include that :)
    Kafka publish is working:
    image
    But I cannot load from Cadence-Web and it's giving an Internal Server Error 400, I tried to use a version of ElastciSearch 7.X and still the same issue. I will try to see what's wrong....

  • Elasticsearch with TLS integration + Kafka with TLS + SASL - Advanced visibility with Elasticsearch TLS
    Blocked temporaly

  • S3 archival - History archival to S3
    I didn't need the env variables for credentials, but if the Bucket have permission restrictions it will be needed.
    image
    image
    This values:
    image
    image
    image

  • GCS archival - Visibility archival to Google Cloud Storage
    I don't have GCP available for the testing. But the schema follows this README:
    https://github.com/cadence-workflow/cadence/blob/v1.3.0/common/archiver/gcloud/README.md
    So it will work correctly.

  • Filestore archival - Local file archival
    Tested with this values:
    image
    image
    image

Monitoring & Metrics

  • Prometheus metrics - Metrics collection with Prometheus
    {"level":"warn","ts":"2025-06-19T12:01:36.325Z","msg":"error in prometheus reporter","service":"cadence-frontend","error":"a previously registered descriptor with the same fully-qualified name as Desc{fqName: "cadence_latency", help: "cadence_latency histogram", constLabels: {}, variableLabels: [cadence_service operation domain caller transport]} has different label names or a different help string","logging-call-at":"metrics.go:151"}
    It's an issue in cadence 1.3.1. Should be fixed in 1.3.2 http://github.com/cadence-workflow/cadence/pull/6963
    Still getting the error msg but metrics are working.
image
  • StatsD metrics - Metrics collection with StatsD
    I don't have a statsD installation so I just check enabling and the schema is reading it, so it's okey.

Advanced Cluster

  • Global TLS volumes - Centralized TLS certificate management
    Validated in Cassandra with TLS check.
  • Global environment variables - Shared env vars across services
    Validated in Secrets handled check and in rest of checks.
  • Dynamic configuration - File-based dynamic config
    Validated changing the default visibility to db from es (elasticSearch) adding the dynamicConfig writeVisibilityStoreName readVisibilityStoreName. You can try yourself to remove it, and you will see that it's used the default "ES" value and cadence won't work with defaults.

@CosminL-DEV
Copy link
Contributor Author

CosminL-DEV commented Jun 23, 2025

Finished DB Compatibility Tests. This is the pod and commands that I used to test it:

apiVersion: v1
kind: Pod
metadata:
  name: cadence-debug-pod
  namespace: cadence-testing
  labels:
    app: cadence-debug
spec:
  restartPolicy: Always
  initContainers:
  - name: fix-cert-permissions
    image: alpine:3.18
    command: ["/bin/sh"]
    args:
      - -c
      - |
        # Copy certificates from secret mount to writable location
        cp /tmp/certs-source/ca.pem /etc/cadence/ssl/ca.pem
        cp /tmp/certs-source/client.pem /etc/cadence/ssl/client.pem
        cp /tmp/certs-source/client-key.pem /etc/cadence/ssl/client-key.pem
        
        # Set correct permissions
        chmod 644 /etc/cadence/ssl/ca.pem
        chmod 644 /etc/cadence/ssl/client.pem
        chmod 600 /etc/cadence/ssl/client-key.pem
        
        # Verify permissions
        echo "Certificate permissions:"
        ls -la /etc/cadence/ssl/
    volumeMounts:
      - name: mysql-tls-certs
        mountPath: /tmp/certs-source/
        readOnly: true
      - name: mysql-tls-certs-writable
        mountPath: /etc/cadence/ssl/
  containers:
  - name: cadence-server
    image: docker.io/ubercadence/server:v1.3.1
    imagePullPolicy: IfNotPresent
    command: ["/bin/sh"]
    args: ["-c", "while true; do sleep 3600; done"]
    env:
      - name: KEYSPACE
        value: cadence
      - name: VISIBILITY_KEYSPACE
        value: cadence_visibility
      - name: RF
        value: "1"
      - name: CADENCE_HOME
        value: /etc/cadence
    volumeMounts:
      - name: mysql-tls-certs-writable
        mountPath: /etc/cadence/ssl/
        readOnly: true
  volumes:
    - name: mysql-tls-certs
      secret:
        secretName: mysql-crt
        items:
          - key: ca.crt
            path: ca.pem
          - key: tls.crt
            path: client.pem
          - key: tls.key
            path: client-key.pem
    - name: mysql-tls-certs-writable
      emptyDir: {}
---
# Cassandra Schema Setup
SCHEMA_DIR=$CADENCE_HOME/schema/cassandra/cadence/versioned
cadence-cassandra-tool --ep cassandra.cadence-testing.svc.cluster.local -u cassandra -pw 1h2GcyDEhU --aa "org.apache.cassandra.auth.PasswordAuthenticator" create -k $KEYSPACE --rf $RF
cadence-cassandra-tool --ep cassandra.cadence-testing.svc.cluster.local -u cassandra -pw 1h2GcyDEhU --aa "org.apache.cassandra.auth.PasswordAuthenticator" -k $KEYSPACE setup-schema -v 0.0
cadence-cassandra-tool --ep cassandra.cadence-testing.svc.cluster.local -u cassandra -pw 1h2GcyDEhU --aa "org.apache.cassandra.auth.PasswordAuthenticator" -k $KEYSPACE update-schema -d $SCHEMA_DIR

VISIBILITY_SCHEMA_DIR=$CADENCE_HOME/schema/cassandra/visibility/versioned
cadence-cassandra-tool --ep cassandra.cadence-testing.svc.cluster.local -u cassandra -pw 1h2GcyDEhU --aa "org.apache.cassandra.auth.PasswordAuthenticator" create -k $VISIBILITY_KEYSPACE --rf $RF
cadence-cassandra-tool --ep cassandra.cadence-testing.svc.cluster.local -u cassandra -pw 1h2GcyDEhU --aa "org.apache.cassandra.auth.PasswordAuthenticator" -k $VISIBILITY_KEYSPACE setup-schema -v 0.0
cadence-cassandra-tool --ep cassandra.cadence-testing.svc.cluster.local -u cassandra -pw 1h2GcyDEhU --aa "org.apache.cassandra.auth.PasswordAuthenticator" -k $VISIBILITY_KEYSPACE update-schema -d $VISIBILITY_SCHEMA_DIR
---
# Cassandra TLS Schema Setup (not working, so I will use without SSL)
KEYSTORE_PASSWORD=$(cat /etc/cadence/ssl/passwords/keystore-password)
TRUSTSTORE_PASSWORD=$(cat /etc/cadence/ssl/passwords/truststore-password)
cadence-cassandra-tool \
  --ep $CASSANDRA_SEEDS \
  -u cassandra \
  -pw ZDlnq3sj9G \
  --aa "org.apache.cassandra.auth.PasswordAuthenticator" \
  --tls \
  --tls-ca-file /etc/cadence/ssl/ca.pem \
  --tls-cert-file /etc/cadence/ssl/client.pem \
  --tls-key-file /etc/cadence/ssl/client-key.pem \
  create -k $KEYSPACE --rf $RF
---
# PostgreSQL Schema Setup
cadence-sql-tool --ep postgresql.cadence-testing.svc.cluster.local -p 5432 -u postgres -pw tdEmPobbF9 --plugin postgres create-database --db cadence
cadence-sql-tool --ep postgresql.cadence-testing.svc.cluster.local -p 5432 -u postgres -pw tdEmPobbF9 --plugin postgres create-database --db cadence_visibility
cadence-sql-tool --ep postgresql.cadence-testing.svc.cluster.local -p 5432 -u postgres -pw tdEmPobbF9 --plugin postgres --db cadence setup-schema -v 0.0
cadence-sql-tool --ep postgresql.cadence-testing.svc.cluster.local -p 5432 -u postgres -pw tdEmPobbF9 --plugin postgres --db cadence update-schema -d ./schema/postgres/cadence/versioned
cadence-sql-tool --ep postgresql.cadence-testing.svc.cluster.local -p 5432 -u postgres -pw tdEmPobbF9 --plugin postgres --db cadence_visibility setup-schema -v 0.0
cadence-sql-tool --ep postgresql.cadence-testing.svc.cluster.local -p 5432 -u postgres -pw tdEmPobbF9 --plugin postgres --db cadence_visibility update-schema -d ./schema/postgres/visibility/versioned
---
# PostgreSQL TLS Schema Setup
cadence-sql-tool --ep postgresql.cadence-testing.svc.cluster.local -p 5432 -u postgres -pw qdeDxXG5Fn --tls --tls-ca-file /etc/cadence/ssl/ca.pem --tls-cert-file /etc/cadence/ssl/client.pem --tls-key-file /etc/cadence/ssl/client-key.pem --plugin postgres create-database --db cadence
cadence-sql-tool --ep postgresql.cadence-testing.svc.cluster.local -p 5432 -u postgres -pw qdeDxXG5Fn --tls --tls-ca-file /etc/cadence/ssl/ca.pem --tls-cert-file /etc/cadence/ssl/client.pem --tls-key-file /etc/cadence/ssl/client-key.pem --plugin postgres create-database --db cadence_visibility
cadence-sql-tool --ep postgresql.cadence-testing.svc.cluster.local -p 5432 -u postgres -pw qdeDxXG5Fn --tls --tls-ca-file /etc/cadence/ssl/ca.pem --tls-cert-file /etc/cadence/ssl/client.pem --tls-key-file /etc/cadence/ssl/client-key.pem --plugin postgres --db cadence setup-schema -v 0.0
cadence-sql-tool --ep postgresql.cadence-testing.svc.cluster.local -p 5432 -u postgres -pw qdeDxXG5Fn --tls --tls-ca-file /etc/cadence/ssl/ca.pem --tls-cert-file /etc/cadence/ssl/client.pem --tls-key-file /etc/cadence/ssl/client-key.pem --plugin postgres --db cadence update-schema -d ./schema/postgres/cadence/versioned
cadence-sql-tool --ep postgresql.cadence-testing.svc.cluster.local -p 5432 -u postgres -pw qdeDxXG5Fn --tls --tls-ca-file /etc/cadence/ssl/ca.pem --tls-cert-file /etc/cadence/ssl/client.pem --tls-key-file /etc/cadence/ssl/client-key.pem --plugin postgres --db cadence_visibility setup-schema -v 0.0
cadence-sql-tool --ep postgresql.cadence-testing.svc.cluster.local -p 5432 -u postgres -pw qdeDxXG5Fn --tls --tls-ca-file /etc/cadence/ssl/ca.pem --tls-cert-file /etc/cadence/ssl/client.pem --tls-key-file /etc/cadence/ssl/client-key.pem --plugin postgres --db cadence_visibility update-schema -d ./schema/postgres/visibility/versioned
---
# MySQL Schema Setup
cadence-sql-tool --ep mysql.cadence-testing.svc.cluster.local -p 3306 -u root -pw voxAAfyTLH --plugin mysql create-database --db cadence
cadence-sql-tool --ep mysql.cadence-testing.svc.cluster.local -p 3306 -u root -pw voxAAfyTLH --plugin mysql create-database --db cadence_visibility
cadence-sql-tool --ep mysql.cadence-testing.svc.cluster.local -p 3306 -u root -pw voxAAfyTLH --plugin mysql --db cadence setup-schema -v 0.0
cadence-sql-tool --ep mysql.cadence-testing.svc.cluster.local -p 3306 -u root -pw voxAAfyTLH --plugin mysql --db cadence update-schema -d ./schema/mysql/v8/cadence/versioned
cadence-sql-tool --ep mysql.cadence-testing.svc.cluster.local -p 3306 -u root -pw voxAAfyTLH --plugin mysql --db cadence_visibility setup-schema -v 0.0
cadence-sql-tool --ep mysql.cadence-testing.svc.cluster.local -p 3306 -u root -pw voxAAfyTLH --plugin mysql --db cadence_visibility update-schema -d ./schema/mysql/v8/visibility/versioned
---
# MySQL TLS Schema Setup
cadence-sql-tool --ep mysql.cadence-testing.svc.cluster.local -p 3306 -u root -pw aYtpr7o8as --tls --tls-ca-file /etc/cadence/ssl/ca.pem --tls-cert-file /etc/cadence/ssl/client.pem --tls-key-file /etc/cadence/ssl/client-key.pem --plugin mysql create-database --db cadence
cadence-sql-tool --ep mysql.cadence-testing.svc.cluster.local -p 3306 -u root -pw aYtpr7o8as --tls --tls-ca-file /etc/cadence/ssl/ca.pem --tls-cert-file /etc/cadence/ssl/client.pem --tls-key-file /etc/cadence/ssl/client-key.pem --plugin mysql create-database --db cadence_visibility
cadence-sql-tool --ep mysql.cadence-testing.svc.cluster.local -p 3306 -u root -pw aYtpr7o8as --tls --tls-ca-file /etc/cadence/ssl/ca.pem --tls-cert-file /etc/cadence/ssl/client.pem --tls-key-file /etc/cadence/ssl/client-key.pem --plugin mysql --db cadence setup-schema -v 0.0
cadence-sql-tool --ep mysql.cadence-testing.svc.cluster.local -p 3306 -u root -pw aYtpr7o8as --tls --tls-ca-file /etc/cadence/ssl/ca.pem --tls-cert-file /etc/cadence/ssl/client.pem --tls-key-file /etc/cadence/ssl/client-key.pem --plugin mysql --db cadence update-schema -d ./schema/mysql/v8/cadence/versioned
cadence-sql-tool --ep mysql.cadence-testing.svc.cluster.local -p 3306 -u root -pw aYtpr7o8as --tls --tls-ca-file /etc/cadence/ssl/ca.pem --tls-cert-file /etc/cadence/ssl/client.pem --tls-key-file /etc/cadence/ssl/client-key.pem --plugin mysql --db cadence_visibility setup-schema -v 0.0
cadence-sql-tool --ep mysql.cadence-testing.svc.cluster.local -p 3306 -u root -pw aYtpr7o8as --tls --tls-ca-file /etc/cadence/ssl/ca.pem --tls-cert-file /etc/cadence/ssl/client.pem --tls-key-file /etc/cadence/ssl/client-key.pem --plugin mysql --db cadence_visibility update-schema -d ./schema/mysql/v8/visibility/versioned

ElasticSearch Schema Manual

apiVersion: v1
kind: Pod
metadata:
  name: cadence-debug-pod
  namespace: cadence-testing
  labels:
    app: cadence-debug
spec:
  restartPolicy: Always
  initContainers:
  - name: fix-cert-permissions
    image: alpine:3.18
    command: ["/bin/sh"]
    args:
      - -c
      - |
        # Copy certificates from secret mount to writable location
        cp /tmp/certs-source/ca.pem /etc/cadence/ssl/ca.pem
        cp /tmp/certs-source/client.pem /etc/cadence/ssl/client.pem
        cp /tmp/certs-source/client-key.pem /etc/cadence/ssl/client-key.pem
        
        # Set correct permissions
        chmod 644 /etc/cadence/ssl/ca.pem
        chmod 644 /etc/cadence/ssl/client.pem
        chmod 600 /etc/cadence/ssl/client-key.pem
        
        # Verify permissions
        echo "Certificate permissions:"
        ls -la /etc/cadence/ssl/
    volumeMounts:
      - name: elasticsearch-tls-certs
        mountPath: /tmp/certs-source/
        readOnly: true
      - name: elasticsearch-tls-certs-writable
        mountPath: /etc/cadence/ssl/
  containers:
  - name: cadence-server
    image: docker.io/ubercadence/server:v1.3.1
    imagePullPolicy: IfNotPresent
    command: ["/bin/sh"]
    args: ["-c", "while true; do sleep 3600; done"]
    env:
      - name: KEYSPACE
        value: cadence
      - name: VISIBILITY_KEYSPACE
        value: cadence_visibility
      - name: RF
        value: "1"
      - name: CADENCE_HOME
        value: /etc/cadence
    volumeMounts:
      - name: elasticsearch-tls-certs-writable
        mountPath: /etc/cadence/ssl/
        readOnly: true
  volumes:
    - name: elasticsearch-tls-certs
      secret:
        secretName: elasticsearch-master-crt
        items:
          - key: ca.crt
            path: ca.pem
          - key: tls.crt
            path: client.pem
          - key: tls.key
            path: client-key.pem
    - name: elasticsearch-tls-certs-writable
      emptyDir: {}
---
# Cassandra Schema Setup
SCHEMA_DIR=$CADENCE_HOME/schema/cassandra/cadence/versioned
cadence-cassandra-tool --ep cassandra.cadence-testing.svc.cluster.local -u cassandra -pw 5Zd7mr2L0H --aa "org.apache.cassandra.auth.PasswordAuthenticator" create -k $KEYSPACE --rf $RF
cadence-cassandra-tool --ep cassandra.cadence-testing.svc.cluster.local -u cassandra -pw 5Zd7mr2L0H --aa "org.apache.cassandra.auth.PasswordAuthenticator" -k $KEYSPACE setup-schema -v 0.0
cadence-cassandra-tool --ep cassandra.cadence-testing.svc.cluster.local -u cassandra -pw 5Zd7mr2L0H --aa "org.apache.cassandra.auth.PasswordAuthenticator" -k $KEYSPACE update-schema -d $SCHEMA_DIR
---
# ElasticSearch Schema Setup
SCHEMA_FILE=$CADENCE_HOME/schema/elasticsearch/v7/visibility/index_template.json
server=`echo elasticsearch.cadence-testing.svc.cluster.local:9200 | awk -F ',' '{print $1}'`
URL="http://$server/_template/cadence-visibility-template"
curl -X PUT $URL -H 'Content-Type: application/json' --data-binary "@$SCHEMA_FILE"
URL="http://$server/cadence-visibility"
curl -X PUT $URL
---
# ElasticSearch TLS Schema Setup
SCHEMA_FILE=$CADENCE_HOME/schema/elasticsearch/v7/visibility/index_template.json
server=`echo elasticsearch.cadence-testing.svc.cluster.local:9200 | awk -F ',' '{print $1}'`
URL="https://$server/_template/cadence-visibility-template"
curl -u elastic:testingPassword --cacert /etc/cadence/ssl/ca.pem --cert /etc/cadence/ssl/client.pem --key /etc/cadence/ssl/client-key.pem -X PUT $URL -H 'Content-Type: application/json' --data-binary "@$SCHEMA_FILE"
URL="https://$server/cadence-visibility"
curl -u elastic:testingPassword --cacert /etc/cadence/ssl/ca.pem --cert /etc/cadence/ssl/client.pem --key /etc/cadence/ssl/client-key.pem -X PUT $URL

@CosminL-DEV CosminL-DEV marked this pull request as ready for review June 24, 2025 14:25
@CosminL-DEV
Copy link
Contributor Author

@taylanisikdemir I just pushed this for review. But still pending 2 points:

  • ElasticSearch: I opened an issue for this problem so it's not available right now, but I thinks it's more like a schema problem, so if it's that the case we can continue with the merge...
  • Prometheus metrics: there is an issue in cadence-server 1.3.1 that creates two reporters so it's getting that error, should be fixed in 1.3.2 http://github.com/cadence-workflow/cadence/pull/6963

Let me know if you need something :)

@taylanisikdemir
Copy link
Member

@taylanisikdemir I just pushed this for review. But still pending 2 points:

  • ElasticSearch: I opened an issue for this problem so it's not available right now, but I thinks it's more like a schema problem, so if it's that the case we can continue with the merge...
  • Prometheus metrics: there is an issue in cadence-server 1.3.1 that creates two reporters so it's getting that error, should be fixed in 1.3.2 http://github.com/cadence-workflow/cadence/pull/6963

Let me know if you need something :)

Thanks. ES issue might need some config changes. Team is looking.

@CosminL-DEV
Copy link
Contributor Author

Fixed metrics error, the msg is still in the log but at least are working...

@CosminL-DEV
Copy link
Contributor Author

Ready to merge @taylanisikdemir

Copy link
Member

@taylanisikdemir taylanisikdemir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good at first glance but since it's a huge change I'd like to test it before merging. I will get back to this next week.

failoverVersionIncrement: {{ .Values.config.cluster.failoverVersionIncrement | default 10 }}
primaryClusterName: {{ .Values.config.cluster.primaryClusterName | default "cluster0" }}
currentClusterName: {{ .Values.config.cluster.currentClusterName | default "cluster0" }}
clusterGroup:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By default it makes sense to populate clusterGroup with just current cluster info. This would be out of the box single cluster deployment.
However, when clusterGroup is populated in values.yml (not unset) then we should just use that here instead of adding current cluster by default. The config would provide complete list of clusters, not just others.

Copy link
Member

@taylanisikdemir taylanisikdemir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I followed the steps in CONTRIBUTING.md and the services didn't startup. Stuck on init.

k logs cadence-release-frontend-5584f8b696-vfq6k -n cadencetest -c wait-for-schema
...
Waiting for Cassandra schema to be ready...
Traceback (most recent call last):
  File "/opt/cassandra/bin/cqlsh.py", line 2354, in <module>
    main(*read_options(sys.argv[1:], os.environ))
  File "/opt/cassandra/bin/cqlsh.py", line 2300, in main
    shell = Shell(hostname,
  File "/opt/cassandra/bin/cqlsh.py", line 468, in __init__
    load_balancing_policy=WhiteListRoundRobinPolicy([self.hostname]),
  File "/opt/cassandra/bin/../lib/cassandra-driver-internal-only-3.25.0.zip/cassandra-driver-3.25.0/cassandra/policies.py", line 425, in __init__
  File "/opt/cassandra/bin/../lib/cassandra-driver-internal-only-3.25.0.zip/cassandra-driver-3.25.0/cassandra/policies.py", line 426, in <listcomp>
  File "/usr/lib/python3.10/socket.py", line 955, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
Waiting for Cassandra schema to be ready...

cassandra:
image:
repository: "cassandra"
tag: "4.0"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have cassandra:4.1.1 by default but schema check is using cassandra:4.0. Is that intentional?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants